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There exist very few results on mixing for non-stationary processes. However, mixing is of- 
ten required in statistical inference for non-stationary processes such as time-varying ARCH 
(tvARCH) models. In this paper, bounds for the mixing rates of a stochastic process are derived 
in terms of the conditional densities of the process. These bounds are used to obtain the a, 2- 
mixing and /3-mixing rates of the non-stationary time- varying ARCH(p) process and ARCH(oo) 
process. It is shown that the mixing rate of the time-varying ARCH(p) process is geometric, 
whereas the bound on the mixing rate of the ARCH(oo) process depends on the rate of decay 
of the ARCH(oo) parameters. We note that the methodology given in this paper is applicable 
to other processes. 

Keywords: 2-mixing; absolutely regular (/3-mixing) ARCH(cx)); conditional densities; strong 
mixing (a-mixing); time- varying ARCH 

1. Introduction 

Mixing is a measure of dependence between elements of a random sequence that has 
a wide range of theoretical applications (see [7] and below). One of the most popular 
mixing measures is a-mixing (also called strong mixing), where the a-mixing rate of the 
non-stationary stochastic process {Xt\ is defined as a sequence of coefficients a{k) such 
that 

a(A:)=sup sup \P{G r\ H) - P{G)P{H)\. (1) 

teZ Hea(Xt,Xt-u...) 
Gez(T(Xt+fc,Xt+fc+i,...) 

{Xt\ is called a-mixing if a{k) — > as fc — > 00. a-mixing has several applications in sta- 
tistical inference. For example, if {a(fc)} decays sufficiently fast to zero as fc — >■ 00, then, 
among other results, it is possible to show asymptotic normality of sums of {Xk} (see 
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[12], Chapter 24), as well as exponential inequalities for such sums (see [4]), asymptotic 
normality of kernel- based nonparametric estimators (see [4]) and consistency of change 
point detection schemes of nonlinear time series (see [16]). The notion of 2-mixing is re- 
lated to strong mixing, but is a weaker condition as it measures the dependence between 
two random variables and not the entire tails. 2-mixing is often used in statistical infer- 
ence, for example, deriving rates in nonparametric regression (see [4]). The 2-mixing rate 
can be used to derive bounds for the covariance between functions of random variables, 
say cov{g{Xt) , g{Xt+k)) (see [24]), which is usually not possible when only the correlation 
structure of {Xk} is known. The 2-mixing rate of {^fc} is defined as a sequence a{k) 
which satisfies 

a(fc)=sup sup \P{Gr\H)- P{G)P{H)\. (2) 

teZ H£a{Xt) 

Ge<j(x-t+k) 

It is clear that a{k) < a{k). A closely related mixing measure, introduced in [39] is /3- 
mixing (also called absolutely regular mixing). The /3- mixing rate of the stochastic process 
{Xt} is defined as a sequence of coefficients /3{k) such that 

/3(fc)=sup sup ^^|P(G,niJ,)-P(G,)P(i?,)|, (3) 

iSZ {Hj}ea(Xt.Xt-i,...) j J 

{Gj}eiT{Xt+k,Xt-i-k+i,---) 

where {Gi} and {Hj} are finite partitions of the sample space i}. {Xt} is called /3-mixing 
if /3(fc) — >■ as fc — >■ oo . It can be seen that this measure is slightly stronger than a- mixing 
(since an upper bound for /3(fc) immediately gives a bound for a{k) due to the fact that 
I3ik)>a{k)). 

Despite the versatility of mixing, its main drawback is that, in general, it is difficult to 
derive bounds for a{k), a{k) and I3{k). However, the mixing bounds of some processes are 
known. Chanda [9], Gorodetskii [20], Athreya and Pantula [1] and Pham and Tran [32] 
show strong mixing of the MA(oo) process. Feigin and Tweedie [13] and Pham [31] have 
shown geometric ergodicity of bilinear processes (we note that stationary geometrically 
ergodic Markov chains are geometrically a-mixing, 2-mixing and /3-mixing; see, e.g., [14]). 
More recently, Tjostheim [38] and Mokkadem [30] have shown geometric ergodicity for 
a general class of Markovian processes. The results in [30] have been applied in [6] to 
show geometric ergodicity of stationary ARCII(p) and GARCII(p, q) processes, where p 
and q are finite integers. Related results on mixing for GARCII(p, g) processes can be 
found in [8, 25, 26, 35] (for an excellent review) and [14, 27] (where mixing of 'nonlinear' 
GARCII(p, 5) processes is also considered). Most of these these results are proved by 
verifying the Meyn-Tweedie conditions (see [13] and [28]) and, as mentioned above, are 
derived under the premise that the process is stationary (or asymptotically stationary) 
and Markovian. Clearly, if a process is non-stationary, then the aforementioned results do 
not hold. Therefore, for nonstationary processes, an alternative method to prove mixing 
is required. 

The main aim of this paper is to derive a bound for (1), (2) and (3) in terms of the 
densities of the process plus an additional term, which is an extremal probability. These 
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bounds can be applied to various processes. In this paper, we will focus on ARCH-type 
processes and use the bounds to derive mixing rates for time- varying ARCH(p) (tvARCH) 
and ARCH(oo) processes. The ARCH family of processes is widely used in finance to 
model the evolution of returns on financial instruments; we refer the reader to the re- 
view article of [18] for a comprehensive overview of mathematical properties of ARCH 
processes and a list of further references. It is worth mentioning that Hermann [23] and 
Berkes et al. [3] have considered a different type of dependence, namely a version of the 
m-dependence moment measure, for ARCH-type processes. The stationary GARCH(p, g) 
model tends to be the benchmark financial model. However, in certain situations, it may 
not be the most appropriate model. For example, it cannot adequately explain the long 
memory seen in the data or change according to shifts in the world economy. There- 
fore, attention has recently been paid to tvARCH models (see, e.g., [11, 15, 16, 29]) 
and ARCH(c)o) models (see [17, 19, 33, 37]). The derivations of the sampling properties 
of some of the aforementioned papers rely on quite sophisticated assumptions on the 
dependence structure, in particular, on their mixing properties. 

We will show that, due to the p-Markovian nature of the time-varying ARCH(p) pro- 
cess, the a-mixing, 2-mixing and /3-mixing bounds have the same geometric rate. The 
story is different for ARCH(oo) processes, where the mixing rates can be different and 
vary according to the rate of decay of the parameters. An advantage of the approach 
presented in this paper is that these methods can readily be used to establish mixing 
rates of several time series models. This is especially useful in time series analysis, for 
example, change point detection schemes for nonlinear time series, where strong mix- 
ing of the underlying process is often required. The price we pay for the flexibility of 
our approach is that the assumptions under which we work are slightly stronger than 
the standard assumptions required to prove geometric mixing of the stationary GARCH 
process. However, the conditions do not rely on proving irreducibility (which is usually 
required when showing geometric ergodicity) of the underlying process, which can be 
difficult to verify. 

In Section 2, we derive a bound for the mixing rate of general stochastic processes, in 
terms of the differences of conditional densities. In Section 3, we derive mixing bounds 
for time- varying ARCH(j3) processes (where p is finite). In Section 4, we derive mixing 
bounds for ARCH(oo) processes. Proofs which are not in the main body of the pa- 
per can be found in the Appendix and the accompanying technical report, available at 
http : / / stats . Ise . ac .uk/f ryzlewicz/mixing/tvARCH_mixing . pdf . 

2. Some mixing inequalities for general processes 
2.1. Notation 

For fc > 0, let = {Xt,..., Xt-k); if fc < 0, then = 0. Let = (y., . . . , yo)- Let 

]] • ]| denote the ^i-norm. Let denote the sample space. The a-algebra generated by 
Xt,..., Xt+r is denoted T^^^ = cr(Xt, . . . , Xt+r). 
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2.2. Some mixing inequalities 

Let us suppose that {Xt\ is an arbitrary stochastic process. In this section, we derive 
some bounds for a{k)^ a{k) and /3(fc). To do this, we wiU consider bounds for 

sup \P{GC]H)- P{G)P{H)\ and 



sup ^|P(G,ni7,)-^(G.)P(iI,)l, 



where T'i,r2 > and {C?i} and {Hi\ are partitions of fi. In the proposition befow, we 
give a bound for the mixing rate in terms of conditional densities. Similar bounds for 
linear processes have been derived in [9] and [20] (see also [12], Chapter 14). However, the 
bounds in Proposition 2.1 apply to any stochastic process and it is this generality that 
allows us to use the result in later sections, where we derive mixing rates for ARCH- type 
processes. 

Proposition 2.1. Let us suppose that the conditional density of X^^^^j, | given X_\~^^ 
exists and denote it as f^t+k ly'-'-i • For rj— (j]q, . . . ,rir,) £ (]R+)''i+"'^, define the set 

E = {oj; X\~^^ (oj) £ £}. where E — {{vq, ... ,Vrx)', for all \vj\ <rij}. (4) 
For all ri, r2 > and rj, we then have 

sup \P{Gf^H)-P{G)P{H)\ 

(5) 



<2sup/ 1/^.+. ,x*~';iy\x)-f^,+. .^.,iy\0)\dy + 4P{E-) 



sup ^|P(G,nH,)-P(GOP(i?,)l 
{H,}eTr^\{G,}eJ^ltk+.2 'J 

(6) 

<2 [ sup|/^.+. ,x^-r,{y\x)-f^,+k .^.-r,{y\0)\dy + 4P{E-), 

where {Gi} and {Hj} are finite partitions of fl. Letting W^'^l, ^ be a random vector that 
is independent of X\~^^ , we then have 

sup \P{Gr\H) - P{G)P{H)\ 

■t + k 

t+k + 7'2 

(7) 

<2VsupEi^( sup f V,^k.tiys\ys^i,W,x)dys] +4.P{E^) 
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and 



sup ^|P(G,ni/,)-^(GO^(i?,)l 

(8) 



<2VEvvf sup f supVs.k.t{ys\ys-i,W,x)dys) +^P{E^), 



where Ew{g{W)) = J g{w)fwiw) <iw, fwim.) is the density of w, 'DQ^k,tiyQ\y-i,w,x) = 
|/s,fc,t(ys|w,x) - /s,fe,t(ys|w,0)| and, for s>l, 

T^s,k,t{ys\ys-i,w,x) = \fs.k.t{ys\ys-i,w,x) - fs.k.t{ys\ys-i,w,o)\ (9) 

with the conditional density of Xt+k given iW\'^^\ denoted fo.k.t, the conditional 

density of Xt+k+s given {2Lltk+s-i^i^ttk-i^2L\~''^) denoted fs^kd, £ = (xq, . . . , X-^J 
and w = (wfc, . . . ,wi). 

Proof. This can be found in Appendix A.l. □ 

Since the above bounds hold for all vectors rj G (IR.+ )'"i+^ (note that rj defines the set 
E; see (4)). by choosing the i] which balances the integral and P{E'^), wc obtain an upper 
bound for the mixing rate. 

The main application of the inequality in (7) is to processes which are 'driven' 
by the innovations (e.g., linear and ARCH-type processes). If W^f^l,_-^ is the inno- 
vation process, it can often be shown that the conditional density of Xt+k+s given 
(■^ttk I c 1 ■ ^tT.l 1 ■ ) can be written as a function of the innovation density. De- 

riving the density of Xt+k+s given 

(^tXfc+s-i'iJ^t+fc-i'^t is not a trivial task, but 
it is often possible. In the subsequent sections, we will apply Proposition 2.1 to obtain 
bounds for the mixing rates. 

The proof of Proposition 2.1 can be foimd in the Appendix, but we give a brief outline 
of it here. Let 

H = W;X\-^^ iu^)en}, G = {^-^K'tXl+r, (^) e G}- (10) 

It is straightforward to show that \P{G n iJ) - P{G)P{H) \ < \P{G n H n E) ~ P{G (1 
E)P{H) \ -\- 2P{E'^). The advantage of this decomposition is that when we restrict X*^''^ 
to the set £ (i.e., not large values of we can obtain a bound for \P{G Ci H Ci E) — 

P{Gr\ E)P{H)\. More precisely, by using the inequality 

inf P(G|X*"'^^ = x)P{H r\E)< P{G r\HnE)<sup P{G\X\-''' = x)P{H n E), 

we can derive upper and lower bounds for P(G OH OE) ~ P{G E)P{H) which depend 
only on E and not H and G, and thus obtain the bounds in Proposition 2.1. 

It is worth mentioning that by using (7), one can establish mixing rates for time- 
varying linear processes (such as the tvMA(oo) process considered in [10]). Using (7) and 
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techniques similar to those used in Section 4, mixing bounds can be obtained for the 
tvMA(oo) process. 

In the foUowing sections, we will derive the mixing rates for ARCH-type processes, 
where one of the challenging aspects of the proof is establishing a bound for the integral 
difFerence in (9). 



3. Mixing for the time-varying ARCH(p) process 
3.1. The tvARCH process 



In [15], it is shown that the tvARCH process can be used to explain the commonly 
observed stylized facts in financial time series (such as the empirical long memory). A 
sequence of random variables {Xt} is said to come from the squares of a time- varying 
ARCH(p) process if it satisfies the representation 



where {Zt} are independent, identically distributed (i.i.d.) positive random variables, 
where ¥,{Zt) = 1 and aj(-) are positive parameters. It is worth comparing (11) with the 
squared tvARCH process used in the statistical literature. Unlike the squared tvARCH 
process considered in, for example, [11] and [15], we have not placed any smoothness 
conditions on the time- varying parameters {%(•)}• The smoothness conditions assumed 
in [11] and [15] are used in order to carry out parameter estimation. However, in this 
paper, we are dealing with mixing of the process, which does not require such strong 
assumptions. The assumptions that we require are stated below. From now on, with a 
slight abuse of terminology, we will call the squared tvARCH process simply the tvARCH 
process. 

Assumption 3.1. (i) For some 5 >Q, sup^gg^^^-^ aj{t) <1~5. 

(ii) inftgz ao(t) > and svLPf^j^ao{t) < oo. 

(iii) Let fz denote the density of Zt- For all a > 0, we have J\fz{u) — fz{u[^ + 
a])| du < Ka for some finite K independent of a. 

(iv) Let fz denote the density of Zf. For all a > 0, we have J supQ<^<^ [/^■(it) — 
fz{u[\ + T\)\du< Ka for some finite K independent of a. 

We note that Assumption 3.1(i)-(ii) guarantees that the ARCH process has a Volterra 
expansion as a solution (see [11], Section 5). Assumption 3.1(iii)-(iv) is a type of Lipschitz 
condition on the density function and is satisfied by various well-known distributions, 
including the chi-squared distributions. We now consider a class of densities which satisfy 
Assumption 3.1(iii)-(iv). Suppose that is bounded, that after some finite point m the 




(11) 
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derivative /' declines monotonically to zero and satisfies J \yf'ziy) \ dy < oo. In this case, 

/>oo 

/ sup \fz{u)~fz{u[l + T])\du 

Jo 0<T<a 



< sup \fziu)-fz{u[l + T])\du+ snp \fziu)-fz{u[l + T])\du 

Jo 0<T<a Jm 0<T<a 

< a(m^ sup \f'ziu)\+ [ u\f'z{u)\du)<Ka 



for some finite K independent of a, hence Assumption 3.1(iii)-(iv) is satisfied. 

Wc use Assumption 3.1(i)-(iii) to obtain the strong mixing rate (2-mixing and a- 
mixing) of the tvARCH(p) process, and the slightly stronger conditions Assumption 
3.1(i)-(ii) and (iv) to obtain the /3-mixing rate of the tvARCH(p) process. We mention 
that in the case that {Xt} is a stationary, ergodic time series, [14] have shown geomet- 
ric ergodicity, which they show implies /3-mixing, under the weaker condition that the 
distribution function of {Zi} can have some discontinuities. 



3.2. The tvARCH(p) process and the Volterra series expansion 

In this section, we derive a Volterra series expansion of the tvARCH process (see also 
[17]). These results allow us to apply Proposition 2.1 to the tvARCH process. We first 
note that the innovations Z_l^^_-^ and are independent random vectors. Hence, 

comparing with Proposition 2.1, we are interested in obtaining the conditional density of 
Xf+k given Z_l'^l._^ and Xd~^^^ (denoted /o,fc,t) and the conditional density of Xt+k+s 

given X^^f.^g_i, Z^X\-i ^"^^ (denoted fs.k.t)- We use these expressions to obtain 

a bound for 'Ds,k,t (defined in (9)), which we use to derive a bound for the mixing rate. 
We now represent {Xt} in terms of {Zt}. To do this, we define 



At 



f ai{t)Zt a2{t)Zt 
1 
1 



ap{t)Zt\ 





V 

(ao(t)^t,0, 





.,0)' 



1 

and 



X 



/ 

t-p+i 



At 



{Xt,Xt-i, 



Using this notation, we have the relation X^^p. 



(ai{t) a2{t) 
1 
1 

V 

iXt-p+i)' . 

rt+k—p 



ap{t)\ 



; 



At+kX^+k-i 



-b^^i,. We note that 



the vector representation of ARCH and GARCH processes has been used in [2, 5, 36] in 
order to obtain some probabilistic properties for ARCH-type processes. Now iterating, 
the relation k times (to get X_'^t+V^^^ terms of A^j^^^^), we have 



Xrt + k—p+l _ 1 

At+fc -O-t+k 



fe-2 

E 



t+k- 



it + k- 



■fe-1 



t+k- 



X 



t-p+1 
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where we set [fliJo ^t+fc-d ^ i-^p denotes the p x p-dimensional identity matrix). We 
use this expansion below. 

Lemma 3.1. Let us suppose that Assumption 3.1(iJ is satisfied. For s >0, we then have 

Xt+k+s = Zt+k+s{Vs,kM) + Qs,k,t{x)}, (12) 

where Z_ ~ Z^t+k' s ~ Q and n > t, we have Vo,k,tiZ_) = ao{t + k) + [At+k x 



J2r=o Yll^i At+k-ibt+k-r-i]i , Qo.k.t{X_) = [A+kYli^i At+k-iK\ ([h denotes 

the first element of a vector). 
For I < s <p, 



rfc-l 



'Ps.k.ti^) = ao{t + k + s) + ^ai{t + k + s)Xt+k+s-i 

i=l 

P 

+ '^ai{t + k + s)Zk+s-i 

i—s 

X < ao{t + k + s — i) 



(13) 



k~\~s—i ( r 

At+k+s-^ ^ <^ n ^* 

+k+s-i-d }bt+k+s~i' 

r=l ld=0 



Qs,kAz,x) 



' k-\-s — i 



^aj(i + fc + s)Zfc+s-t4t+fc+s-i'^ J]^ Atj^k+s-i-dX*^t 



d=0 



and for s > p, we have Vs,k,tiZ_) = ao(t + fc + s) + J2i=i ai{t + k + s)Xt+k+s-i and 
Qs,k,t{Z_,2L) = 0- VVe note that Ps.fc.t(') o,nd Qs,fc,t(') o,re positive random variables and 
for s > 1, Vs,k,ti') is a function of Xf^^,^^_^ (but this has been suppressed in the nota- 
tion). 



□ 



Proof. This is found in Appendix A. 2. 

By using (12), we now show that the 
Z^t+k-i ^^"^ jg function of the density of Zt+k+s- It is clear from (12) that 

Zt+k+s can be expressed as Zt+k+s = -p t{z)'+Q^''k t{z x) ' Therefore, it is straightforward 
to show that 



By using (12), we now show that the conditional density of Xt+k+s given X*"^^^ | „ , 



fs,k,tiys\ys-i,z,x) ■ 



1 



-J- 



'Ps,k,tiz) + Qs,k,t {Z, X) \ Vs,k,t {z) + Qs,k,t {Z, X) 



Vs 



(14) 
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3.3. Strong mixing of the tvARCH(p) process 



The aim of this section is to prove geometric mixing of the tvARCH(p) process without 
appeaUng to geometric ergodicity. Naturally, the results in this section also apply to 
stationary ARCH(p) processes. 

In the following lemma, we use Proposition 2.1 to obtain bounds for the mixing rates. 
It is worth mentioning that the techniques used in the proof below can be applied to 
other Markov processes. 

Lemma 3.2. Suppose that {Xt} is a tvARCH process which satisfies (11). For any 
rj = {rjQ , . . . , ) G (R^ )^ , we then have 



sup \P{GnH)~P{G)P{H)\ 
<2Y]sup/E2_( sup Vs,k,tiys\ys-i,Z_,x)dys] (15) 



p-1 



(16) 



sup ^|P(G,nff,)-P(G,)P(ff,)| 
{H,}e.F--,{G,}e£'+'' ,j 

<2V'supEz_ sup / supT>s^k,t{yAys-i,Z_,x)dyA 
p-1 

+ 4^P(|Xt_,|>ry_,+i), 

3=0 

where z_= (zi, . . . , Zfc-i) and {G^} and {-ffj} are partitions offl and Ez(g(^)) = J g{s.) x 

nJ=iVz(2»)dzj. 

Proof. This can be found in Appendix A. 2. □ 

To obtain a mixing rate for the tvARCH(p) process, we need to bound the integral 
in (15), then obtain the set E which minimizes (15). We will start by bounding 2?s,/c,f, 
which, we recall, is based on the conditional density fs,k,t (defined in (14)). 



Lemma 3.3. Let I?s,fc,t and Qs,k,t be defined as in (9) and (13), respectively. 
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(i) Supposing that Assumption 3.1(i)-(iii) holds, then for all x€ (K'^)p, we have 

-V2/,-ieM=y - / mit&aQit) 

<K{l-5f\\xl 

where K is a finite constant and Q <6 <5 < \ (5 is defined in Assumption 3.1(i) 

(ii) Supposing that Assumption 3.1(i)-(ii) and (vi) hold, then for any set £ (defined 
as in (4)), we have 

VEzf sup [ &nvV,^kAys\ys-uZ,x)dy^<su^K{l-5f\\x\\. (18) 

Proof. This can be found in Appendix A. 2. □ 

Wc now use the lemmas above to show geometric mixing of the tvARCH process. 
Theorem 3.1. (i) Supposing that Assumption 3.1(i)-(iii) holds, then 

sup \P{GC]H)-P{G)P{H)\<Kq^. 
Ge(T(x*+'°) 

(ii) Supposing that Assumption 3.1(i)"(ii) and (iv) hold, then 

sup n ifj) - P{G,)P{Hj)\ < Ka'' 

{H,}ea(2f;°°) J J 



for any \/l — 5 < a <1, where K is a finite constant independent oft and k. 

Proof. We win use (15) to prove (i). Equation (17) gives a bound for the integral dif- 
ference in (15); therefore, all that remains is to bound the probabilities in (15). To do 
this, we first use Markov's inequality, to give X]j=o — V-j) ^ Sj=o ^\^t-j\V-j ■ 

By using the Volterra expansion of Xt (see [11], Section 5), it can be shown that 
supjgg^ElA^tl < (supjg^ao(t))/(l — supjgx ^^^-^ aj(t)). Using these bounds and substi- 
tuting (17) into (15) gives, for every r] S (M+)^', the bound 



sup \P{G nH)- P{G)P{H)\ < 2 \ ' + 4A-^ — , 

H^a(X-t°°) 
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We observe that the right-hand side of the above is minimized when rj^j = (1 — (5)*^^^ (for 
< j < {p — 1)), which gives the bound 

sup \P{G nH)- P{G)P{H)\ < K\l (1 - ~5Y. 

Since the above is true for any < S < S, (ii) is true for any a which satisfies y^l — S < 
a < 1; thus giving the rcsuU. 

To prove (ii), we use an identical argument, but using the bound in (18) instead of 
(17). We omit the details. □ 

Remark 3. 1 . We observe that K and a defined in the above theorem are independent 
of t. Therefore, under Assumption 3.1(i)-(iii), we have a{k) < Ka^ (a-mixing, defined 
in (1)) and under Assumption 3.1(i)-(ii) and (iv), f3{k) < Ka*' (/3-mixing, defined in (3)) 
for all 5 < a < 1 . 

Moreover, since a{Xt+k) C a{Xt+k, ■ ■ . ,Xt+p_i) and (T{Xt) C a{Xt, . . .,Xt-p+i), the 
2-mixing rate is also geometric with a{k) < Ka^ {ct{k) defined in (2)). 

4. Mixing for ARCH(oo) processes 

In this section, we derive mixing rates for the ARCH(oo) process. We first define the 
process and state the assumptions that we will use. 

4.1. The ARCH(oo) process 

The ARCH(oo) process has many interesting features, which are useful in several appli- 
cations. For example, under certain conditions on the coefficients, the ARCH(oo) process 
can exhibit 'near long memory' behaviour (see [17]). The squares of the ARCH(oo) pro- 
cess satisfy the representation 

Xt^Zt(^o+Y/ioXt-^, (19) 

where Zt are i.i.d. positive random variables with lE(Zt) = 1 and aj are positive param- 
eters. With a slight abuse of terminology, we will call the squared ARCH(oo) process 
an ARCH(oo) process. It is worth mentioning that the GARCII(p, 5) process has an 
ARCH(oo) representation, where the aj decay geometrically with j . Giraitis and Robin- 
son [19], Robinson and Zaffaroni [34] and Subba Rao [37] consider parameter estimation 
for the ARCH(oo) process. 

We will use Assumption 3.1 and the assumptions below. 

Assumption (i) We have J^JLi aj <l — S and uq > 0. 
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(ii) For some v>l, E|Xt|'' < oo (we note that this is fulfilled if [IE|^o 11^^'' E°li < 

Giraitis et al. [17] have shown that under Assumption 4.1(i), the ARCH(oo) process has 
a stationary solution and a finite mean (i.e., sup(g2^(^t) *^ It is worth mentioning 
that since the ARCH(oo) process has a stationary solution, the shift t plays no role 
when obtaining mixing bounds, that is, supQ^^(^Xk+t),H£<j{Xt) \P{G<^H) — P{G)P{H) \ = 
svL])Qfzcr[Xk),H&a{Xa)\P{G r\ H) — P{G)P{H)\. Furthermore, the conditional density of 
Xt+k given Z^iW_i and is not a function of t. Hence, in the section below, we let 

/o,fc denote the conditional density of Xt+k given 

{Z!tXk-i and Xr^'^) and for s > 1, let 
fs.k denote the conditional density of Xt+k+s given and 



4.2. The ARCH(oo) process and the Volterra series expansion 



We now write Xk in terms of Z},_i and X — (Xo, Ar_i, . . .) and use this to derive the 
conditional densities /o,fc and fs,k- It can be seen from the result below (equation (20)) 
that, in general, the ARCH(oo) process is not Markovian. 



Lemma 4.1. Suppose that {Xt} satisfies (19). Then 

Xk = Vo.kiZ)Zk + Qo,kiZ,X)Zk, 



(20) 



where 



VoAz) 



Qo,k{z,2L) 



ao 



E E 



n 



^m — 1 



m=l k=j^>--->ji>0 \ i=l 



E E E n«... 

r=l \ m=l k=jm>--->jl=r \ i=l 



n 

m-1 ^ 



(21) 



dr{X). 



Furthermore, setting Qo,fc = for k>l, we have that Qo^kiZ_,2L) satisfies the recur- 
sion QQ.k{Z_,2L) = Y.^j=iajQo,k-j{Z_,X)Zk-j + dk{2L), where dk{X) = Y.jLo^'k-i-j^-j 
(for k>l). 

Proof. This can be found in Appendix A. 3 of the technical report. 



□ 

We will use the result above to derive the 2-mixing rate. To derive a and /3 mixing, 
we require the density of Xk+s given A^^_|_5_^, Z_l_i and 2Lo°° , which uses the following 
lemma. 

Lemma 4.2. Suppose that {Xt} satisfies (19). For s>\, we then have 

Xk+s = Zk+s{VsAz) + QsAz,x)}, 
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s oo 

where Vs,kiZ_) +^ajXk+s-j + ^ ajZh+s-]'Po,k+s-]iZ_), (22) 

J = l j=s+l 

k+s 

Qs,k{Z_,X)= ^ ajZk+s-jQo,k+s-j{Z_TX) + dk+s{]Q- 

3=s+l 

Proof. This can be found in Appendix A. 3 of the technical report. □ 

Using (20) and (22), for all s > 0, we have that Z^+s = -p k(Z)+Q\{z x) ' "^hich leads 
to the conditional densities 

fs,k{ys\ys-i,z,x)^- — / — — TTTT^ — 1 tV (2^) 

T^sM^J + Qs,k[Z,X) \Vs,k{z) + Qs^k[z,x) J 

In the proofs below, Qo,/c(ifc_i,^) plays a prominent role. By using the recursion in 
Lemma 4.1 and (23), setting x = X^°" and noting that '&{Qs,k{Z_,x)) = Qs,fc(ifc_i,^), 
we obtain the recursion Qo,fe(l a;_i,£) = H'j=i ai+s2o,fc-i(lfe-j-i,2) + dk+s{x). We use 
this to obtain a solution for Qo.fed/c-ij 31) in terms of {dk{x)\k in the lemma below. 

Lemma 4.3. Suppose that {Xt\ satisfies (19) and Assumption 4-1 is fulfilled. There 
then exists {V'j} such that for all\z\<l, we have (1 — X^jli cijz^)^^ — Si=o ^i-^"' • Fur- 
thermore, if J2j \ \ < then [22] have shown that < oo. For k <0, set 
dk{x) = and Qo,kiXk-i^2L) — 0. For k>l, Qo,fe(ifc-i7^) then has the solution 

oo k~l k—1 f oo ^ 

So,fe(lfc-i,£) = ^V'jdfe-j(£) ^'^i'jdk-jix) = ^ Vj<^ ^afc_j+ja:_j L (24) 

j=0 j=0 j=0 Ki=0 J 

where x = (xq, . . .). 

Proof. This appears in Appendix A. 3 of the technical report. □ 
4.3. Mixing for ARCH(c») processes 

In this section, we show that the mixing rates are not necessarily geometric and depend 
on the rate of decay of the coefficients {uj} (we illustrate this in the following example). 
Furthermore, for ARCH(oo) processes, the strong mixing rate and 2-mixing rate can be 
different. 

Example 4-1 ■ Let us consider the ARCII(c>d) process, {Xj}, defined in (19). Giraitis et 
al. [17] have shown that if ~ j'^^^^^ (for some 5>0) and [E(Z2)]i/2 ^ a, <1, then 
I cov(Xo,Xfe)| ~ k~'^^^^\ That is, the absolute sum of the covariances is finite, but 'only 
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just' if S is small. If < 1, it is straightforward to see that Xt is a bounded random 
variable and by using Ibragimov's inequality (see [21]), we have 

\coviXo,Xk)\<C sup \P{AnB)-P{A)PiB)\ 

Aea{Xa),B£a{Xk) 

for some C <oo. Noting that \ cov{Xo,Xk)\ = 0{k~'^^+^^), this gives a lower bound of 
0(fc~*^^"'"''') on the 2-mixing rate. 

To obtain the mixing rates we will use Proposition 2.1, this result requires bounds on 
T^sM = \fs,k{ys\ys-i,z,x) - fs,k{ys\ys-i,z,0)\ and its integral. 

Lemma 4.4. Suppose that {Xt} satisfies (19) and let Vg^k o,nd Qo,A;(') be defined as in 
(9) and (21), respectively. IJ Assumptions 3.1(iii) and ^.1 are fulfilled, then 



% / \foAy\z,x)~fo,k{y\z,o)\dy 



^ Qo,fc(lfc-i,£) Y^|,|/v" 

< = I V'i I < 2^ ak-j+^ 

° j=Q 1 1=0 



and, for s>l. 



Ez_\ sup 'Ds,k{ys\ys-i,Z_,x)dy, 



^ ( k+s k+s—j oo oo ^ 

<— < X! °J X! \M^a,k+s-j-l+^X-^+^ak+s+^X-^\ 

If Assumptions 3.1(iv) and 4-1 are fulfilled and £ is defined as in (4), then 



Ez_\ sup sup'Ds,k{ys\ys~i,Z_,x)dys] 



k+s k+s—j 



,j = s+l 1=0 1=0 i=0 ) 



(25) 



(26) 



(27) 



ao 

w/iere X = (xo, X-i, . . .) is a positive vector. 

Proof. This can be found in Appendix A. 3 of the technical report. □ 



We require the following simple lemma to prove the theorem below. 
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Lemma 4.5. If {ci}, {d,} and {rj-t} are positive sequences, then 

( oo ^ oo 

inf ^(c.7y_. +d,7yzr) = {^^K^^'^) ^ ^-^/i^+^))Y^c':/^^+^U\'^''+'\ (28) 



- k 1=0 



Proof. This appears in Appendix A. 3 of the technical report. □ 

In the following theorem, we obtain a-mixing and /3-mixing bounds for the ARCH(oo) 
process. 

Theorem 4.1. Suppose that {Xt} satisfies (19). 

(a) Suppose Assumptions 3.1(iii) and ^.l hold. We then have 

sup \P{Gr\H)- P{G)P{H)\ 

OO A;+s /c+s— j 



— ^ ^ ^ \tPi\ak+s~j^i+i (29) 



an 

" s=0 j=s+l 1=0 



-| 1^/(1^+1) 



i=0 

''oh, 

where K{v) = 3(i/i/(i+'^) + zy-''/(''+i)). 

(i) // the parameters of the ARCH(cxj) process satisfy \aj \ ^ j^^ and li/jj] ^ j^^ 
(^j defined in Lemma 4-3), then we have 

sup \P{GnH)-P{G)P{H)\<K-[k{k + l)-"^+^ + {k + l)-"^+'^], 
where S — 5 x (77^). 

(ii) // the parameters of the ARCH(cxj) process satisfy \aj \ ^ and tpj ~ 5^ , where 
Q <5 < \ (an example is the GARCH(p, q) process), then we have 

sup \P{GC^H)-P{G)P{H)\<G 

where C is a finite constant. 
(b) If Assumptions 3.1(iv) and 4.I hold, then we have 

sup EEl^(^^'^^^)-^(^'')^(^^-)l 



00 k-\-s k-\-s—j 



^ 00 k~\-s k-\-s—j 

— E E E li^iWk+s-j-i+i (30) 



an 

" s=Qj=s+l 1=0 
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-E' 



where {Gi} anrf {^^j} are partitions of fi. W^e mention that the bounds for the 
a-mixing rates for different orders of {aj} and {tpj} derived in (i) also hold for 
the (3-mixing rate. 

Proof. Wc first prove (a). We use the fact that 

sup \P{GnH)-P{G)P{H)\= lim sup H e To°°\P{G H) - P{G)P{H)\ 

and find a bound for each n. By using (5) to bound supg^^pfc HeT^°^ \P{GC\ H) — 
P{G)P{H)\, we see that for all sets £ (as defined in (4)), we have 

sup \P{Gr\H)-P{G)P{H)\ 



<2supY]Ez_( sup <^ 'Ds.k{ys\ys-i,Z_,x)dys 



(31) 



+ 4P(Xo > rio or, . . . , > ?7_„). 
To bound the integral in (31), we use (26) to obtain 

supV'E^I sup / 'Ds,k{ys\ys-i,Z_,x)dys 

71 I' k~\-s k+s—j OO OO 

= — El E E Hll^ak+s-j-l+tV-t+^Ok+s+iV-r 



s=0 kj=s+l (=0 



Now, by using Markov's inequality, we have that P{Xo > r/o or, ...,X_„ > rj^n) < 
X]"=o ^^H^^^- Substituting this and the above into (31) and letting rt — > oo gives 



sup \P{GnH)^ P{G)P{H)\ 



<inf 



OO ^ k+s k+s~j OO 

£Ei E E iV';iE«^-+--^- 



ao 



(32) 



s=0 (j=s+l 1=0 



■ ^ ak+s+iV-i 
i=0 



-4E|Xor£r?Zr 



i=0 



where = (770, 77-1, .. .)■ 
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We now use (28) to minimize (32), which gives us (29). The proof of (i) can be found 
in the technical report. It is straightforward to prove (ii) using (28). 

The proof of (b) is very similar to the proof of (a), but uses (27) rather than (26). We 
omit the details. □ 

Remark 4-1- Under the assumptions of Theorem 4.1(a), we have a bound for the a- 
mixing rate, that is, a{k) < C(fc), where ({k) = K[-^ Y.T=o Ejis+i »j J2'i=o~^ \iJi\ak+s-j-i+i - 
^ X]^o '^fc+s+i]"^''^^^'' ■ Under the assumptions of Theorem 4.1(a), the /3-mixing coeffi- 
cient is bounded by /3(fc) < C{k)- 

In the following theorem, we consider a bound for the 2-mixing rate of an ARCH(oo) 
process. 

Theorem 4.2. Suppose that {Xt} satisfies (19) and that Assumption 3.1(iii) and 4-1 
hold. We then have 

sup \P{GnH)~ P{G)P{H)\ 

G(ia{Xk),H^y^a°° 

(33) 



< 



^ k-l 

ao — 



3=0 



where K{v) ^ 3(j^i/(i+'") + j^-'^/^'^+i)). 

If the parameters of the ARCII(oo) process satisfy aj ^ and lipj] ^ (^j defined 
in Lemma 4-3), then we have 

sup \P{GnH)-P{G)P{H)\<K-k{k + l)-~^+'^, (34) 
where 5 ^ S x (— ^). 

Proof. We use a similar proof to that of Theorem 4.1. The integral difference is replaced 
with the bound in (25) and we again use Markov's inequality: together they give the 
bound 

sup \P{Gr\H)-P{G)P{H)\ 

G&c7{X„),H&J'a°° 

^A — 1 (OO ^ oo ^ 

Finally, to obtain (33) and (34), we use (35) and a proof similar to that of Theorem 
4.1(i). We omit the details. □ 
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Remark 4-2. Comparing (34) and Theorem 4.1(i), we see that the 2-mixing bound is 
of a smaller order than the strong mixing bound. 

In fact, it could well be that the 2-mixing bound is of a smaller order than Theorem 
4.2(i). This is because Theorem 4.2(i) gives a bound for s\rpQ^^^^Xt).H<^(y(XQ,X-i....) |^(Gn 
H) — P{G)P{H)\, whereas the 2-mixing bound restricts the cr-algcbra of the left tail to 
a{Xo). However, we have not been able to show this and this is a problem that requires 
further consideration. 



Appendix: Proofs 

A.l. Proof of Proposition 2.1 

We will use the following three lemmas to prove Proposition 2.1. 

Lemma A.l. Let G € J'ltk+r, = f^(^tXfe+rJ H,Ee TI""^ = cr(X*~''i) (where E 
is defined in (4)), and use the notation of Proposition 2.1. We then have 

\P{G nHHE)- P{G n E)P{H)\ 

<2P(if)sup|P(G|X*r^ ^x)-P{G\x!r-^Q)\ (36) 

+ inf P(G|X*-''i ^x){P{H)P{E'') + P{HC^E^)}. 
Proof. To prove the result, we first observe that 



P{GnHnE)^ P(Zt+fc+,, e G,x\-'-' e{nn £)) 



f f dF(X*--<y,X*t^+,,<x) 
JHn£ Jg 

I I / AP{X\\\_,^^ < y\X!r' = x) ] dP(X*-- < x) 

I ^'(z^t., e Q\x!r^ = x) dP{x!r^ < x). 

JHn£ 



Therefore, by using the above and the fact that P{Hr\E) < P{H), we obtain the following 
inequalities: 

inf e g\x'r' =x)PiHnE) 

~ , (37) 

<p{GnHnE)<snp P{x\+^k+r, e G \x\-^' ^x)P{H) 

x_££ 
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and 



inf P(X*+t+., e^l^'r'^^ =x)P{E) 

~ , (38) 



Subtracting (37) from (38) and using P{Hr\E) = P{H) - PiHnE") gives the inequalities 
P{G OHOE)- P{G n E)P{H) 



< sup P(X*+t+., e g\X\7^^ ^ x)P{H) (39) 



inf P(X*+t+,., e Q\x!r' ^x)P{H)+P{E'^^)P(H), 

P{G OHOE)- P{G n E)P{H) 

> mf P(X!+l^^^ e g\X!r^ ^ x)P{H) (40) 

- supP(rit.+,., e giX*---^ = x)F(ff) - P(i?= n H). 



Combining (39) and (40), wc obtain 



\P{Gc^Hc^E)- P{Gr\E)P{H)\ 

< P{H) supP(G|X*-'"^ =x)^ inf P(G|X*-''i = x) (41) 
+ inf P(G|X*"''i = x){P{H)P{E'=) + P{H E")}. 

Using the triangle inequality, we have 

supP(G|X*-''i =x)- inf P(G|X*'''i =x) < 2sup|P(G|X*-''i = x) - P(G|X*-''i = 0)|. 

xe£ xe£ 

Substituting the above into (41) gives (36), as required. □ 

We now obtain a bound for the first term on the right-hand side of (36). 

Lemma A. 2. Let f„t+k denote the density of | ^ given X\^^'^ and Q and 

H be defined as in (10). Then, 

\P{G\X'r- -£) -P(G|X*r'-^ -0)1 < / I?o,;o,t(yk)dy. (42) 

Jg 
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Let W^*"!'}. ^ be a random vector which is independent of and let fw denote the 

density ofW^^ll_^. If G ecr{Xt+k), then 

and ifGeaiXWl+r^), 

then 

Jg — t+fc+rjl^t — ii^t+fc+rjl— ' — ^ 

<V]Ei^ sup / 2?s,fc,t(ys|ys_i,u),x)dy, 



(44) 



Proof. The proof of (42) is clear from the definition of I?s,fc t, hence we omit the details. 

To prove (43); we first note that by independence of and X\~^ ^ . we have that 

/^l Y*---! (wzl^.) — fw(.w), where f-^^^x'^^'"^ ^^"^ conditional density of given 
X*-'"! . Therefore, we have 



fx,^^\x'-^^^y\^^ / /x,^jwx*-'-i(2^l^'2)/i^(^)^^= / /o,fc,t(yKx)/w(w)dw. 

Substituting the above into L 1/^ i v'-'-i (yls) ~ /v- , ^i-rj (?/|0)| dy and using the def- 
inition of Eyy now gives (43). 

To prove (44), we note that, by using the same argument used to prove (43), we have 

f^t+k |^t-'-i(2/k)= / fw{w)Y\ fs,k,tiys\ys-i,w,x)dw. (45) 

Now, repeatedly subtracting and adding fs.k.t gives 

fx\\i^ \x'r^(y\^)-fx\x\^ ix'r^^y}^) 



= X1 / /iv(w)< ]T/a,fc,t(2;a|ya-l,W,£) |> 

X S n fb,k,tiyb\yb-i,w,0) ^ 

U=s+1 J 

X {fs,kAys\ys-i,w.,x) ~ /s,fc,t(ys|ys-i,w,0)}dw. 

Therefore, taking the integral of the above over Q gives 

t+fc + r2 ' i t-\-k-j-r2 * 



(46) 
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"s-l 



r2 „ ( [s-l f 

-X] / fw(w){ TT / fa,k,t{ya\ya-l,W,x)dya 

Yl / fb,k,t{yb\yb-l,W,x)dyb 



X 



(47) 



X sup / \fs,k.t{ys\ys-i,w,x)-fs,k,tiys\ys^i,w,0)\(iys?dw. 

Vs-iJg, J 

Next, we observe that since Qj C M and ^i^fs.k.t{ys\ys~iiW_,x)dys = 1, we have 
(na=o/e„/a,fc,t(ya|ya-i,w,x)dya)(n&Ls+i/e, Afc,t(y6|y6-i,w^ < 1- Fhially, Sub- 

stituting this bound into (47) gives (44). □ 

The following lemma will be used to show /3-mixing and uses the above lemmas. 

Lemma A. 3. Suppose that {Gi} G ^l+k+n' {^ii ^ -^t"'^ ^'^'^ i^i) '^'^'^ {^j} '^''"^ P**^" 
titions ofVL. We then have 

J2 \P{G^ n H, nE)- P{G, n e)p{h,)\ 

(48) 

< 2y sup|P(Gi|X*-^''i =0;) -P(Gi =0)1 +2P(i;") and 

^ |p(Gi n Hj n i;") - p(g^ n e'')p{Hj)\ < 2P{e''). (49) 

Proof. Substituting the inequality in (36) into Y.^ ^ \P{Gir\ Hj E) - P{Gir\ E)P{Hj)\ 
gives 

\PiG^ n iJ, n ^) - P(G, n e)P{h,)\ 

< 2 ^ P(i/, ) ^ sup |P(G, |X = £) - P(G. IX*"''^ =0)1 (50) 
+ ^ inf P(G.|X*-'-^ =£){P(iJ,)P(ii;^) + P(iJ, n E-)}. 

The sets {Hj} are partitions of n, hence J^iPi^j) = 1 and J^iPi^j n £'=) < 1. Using 
these observations together with (50) gives (48). 

Inequality (49) immediately follows from the fact that {Hj} and {Gi} are disjoint 
sets. □ 



Using the above three lemmas, we can now prove Proposition 2.1. 
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Proof of Proposition 2.1, equation (5). It is straightforward to show that 



\PiGnH)- P{G)P{H)\ < \P{Gr\Hr\E)- P{Gr\E)P{H)\ 

+ |P(G DHDE'')- P{G n E'')P{H)\. 

Now, by substituting (42) into (36) and using the above, we get 

\P{Gr\H)-P{G)P{H)\<2snv f |/^.+. (yk) - /^*+. .-.,{y\0)\dy 

+ inf P(G|X*-'^i = x){P{H)P{E'') + P{H n E'')} 
+ P{G C] H C] E") + P{G C] E'')P{H). 

pr2 + l 



Finally, by using the facts that Q C , P{G n H n E'=) < PiE''), P{G n E'')P{H) < 

rt- 



P{E'') and inf^ef P{G\X*:f'"' = a;) < 1, we obtain (5). □ 



Proof of Proposition 2.1, equation (6). It is worth noting that the proof of (6) is 
similar to the proof of (5). Using (48) and the same arguments as those in the proof of 
(5), we have 

^iP(G,nij,)-m)m.)i 

<2^sup / 1/^.+. (yla) - |^*-.i(|;|0)|dy + 4P(i?^) 

(51) 

<2^ / sup|/^.+. ,^.-rMx)-f^,+, *-.ai|0)|dy + 4P(i?^) 
<2 j sup|/^.+. i^f'-i (yls) - |^*-..i(y|0)|dy + 4P(ii;^), 

where Hj = {wiX*""! (w) G -Hj} and = {w; (w) G ^^^}, which gives (6). □ 

Proof of Proposition 2.1, equation (7). To prove the result, we substitute the bound 
in (44) into (5) to obtain (7). □ 



Proof of Proposition 2.1, equation (8). To prove (8), we substitute (44) into (6) to 
obtain (8). □ 
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Proof of Lemma 3.1. We first prove (12) with s = 0. Suppose that fc > 1. Focusing on 
the first element of X*"^^"^^"^ in (12) and factoring out Zt+k gives 



Xt+k = Zt+kl ao(i + fc) + 



fc-2 r 



At+k n ^*+'^-' iZ)bt+k-r-i{Z) 



r=0 i=l 



A,+k{YlA+k-^(.z) \xY^+' 



. i=l 



which is (12) (with s = 0). To prove (12) for 1 < s < p, we note that using the tvARCH(p) 
representation in (11) and (12) for s = gives 



Xt+k+s = Zt+k+s< aQ{t + k + s) + ^aj(t + fc + s)Xt+k+s-i + ^ai(t + fc + s)Xt+k+s-i 

K. i—1 i—s ) 

= Zt+k+s{VsMz) + QsMz,x)}, 

where 'Ps,k,t and Qs,k,t are defined in (13). Hence, this gives (12). Since aj{-) and Zt are 
positive, it is clear that Vs.k.t and Qs,k,t are positive random variables. □ 

Proof of Lemma 3.2. We first note that since {Xt} satisfies a tvARCH(p) represen- 
tation [p < oo) it is p-Markovian, hence for any r2 > p, the a-algebras generated by 
X^t+k+r2 ^^'^ (:^t+fe+r2'— t+fc+p-i) ^'"'^ "^^^ same. Moreover, by using the fact that for all 
T > Zt is independent of Xt, we have 



sup \P{GnH)- P{G)P{H)\ 

sup \P{Gr\H)- P{G)P{H)\. 



(52) 



Now, by using the above. Proposition 2.1, equation (7), and the fact that and 
^t-p+i g^^g independent, for any set £ (defined as in (4)), we have 



sup 



\P{Gr\H)-P{G)P{H)\ 



2supY]Ez sup 'Ds.k,t{ys\ys-i,z,x)dys] 
+ 4:P{Xt > 770 or, ... , Xt-p+i > V-p+i)- 



< 



(53) 
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Finally, using the fact that P{Xt > rjo or Xf_i > rj^i, . . . , Xt-p+i > 77_p+i) < 
E^Io^(^*-. >^-.) gives (15). 

The proof of (16) is similar to the proof above, but uses (8) instead of (7), so we omit 
the details. □ 

We require the following simple lemma to prove Lemmas 3.3 and 4.4. 

Lemma A. 4. If Assumption 3.1(iii) is satisfied, then, for any positive A and B , we have 

1 „ f y \ I . f y\ , . „fB B 



-J. 



A + B-^'KA + B 



fz 



dy<K 



A A + B 



(54) 



If Assumption 3.1(iv) is satisfied, then, for any positive A, positive continuous function 
B :W^^^^ — > R and set E (defined as in (4)), we have 



sup 



1 



-Jz 



V 



1 



A + B{x)' \A + B{x) 
Proof. To prove (54), we observe that 



dy < K sup 



B{x) B{x) 



A A + B{x) 



(55) 



-J- 



A + B-' \A + B 



A 



A 



dy = I + II, 



where 



1 



A + B 



fz 



y 



A + B 



!z 



dy and // : 



1 



1 



A + B A 



fz 



To bound /, we note that by changing variables with u = y / {A + B) and under Assump- 
tion 3.1(iii), we get 



fziu)-fz[u[l + 



du<K- 



.B 
A' 



It is straightforward to show that // < . Hence, the bounds for / and // give (54). 

The proof of (55) is the same as above, but uses Assumption 3.1 (iw) instead of As- 
sumption 3.1(iii), so we omit the details. □ 



Proof of Lemma 3.3. We first show that 



[ K 

sup / 'Ds,k,t{Vs\ys-i,z,x)dys < — -rxQs,k,t{z,x) 

,_iGK=J - imtezao(i) 



(56) 



and use this to prove (17). We note that when x = 0, Qs,k,t{^, 0) = and fs,k,t{ys\ys-i,z_, 0) 
'Ps,kAs)~^fz{ -p^l'^(^) )- Therefore, using (14) gives 



T^s,kAys\ys-i,z,x) 



:f. 



'Ps,k,t{z) + Qs,k,t{z,x) \'Ps.k,t{z) + Qs,k,t{z,x) 
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Now, recalling that Vs,k,t and Qs,k,t are both positive and setting A = 'Ps,k,t{z), B 
Qs,k,t{z,x) and using (54), we have 



T^s,k,tiys\ys-i,z,x) dys < K 



J Qs,k,ti2.^^) 



Qs.k.t{z,x) 



'Ps,k,t {z) Vs^k.t (z) + Qs,k,t {z, x) 



Finally, since Vs.k.t{^ > inftgzao(t), we have ^^'Ds^kAVs\ys-i,z,x)dys < K^^'^^^^^, 
thus giving (56). By using (56), we now prove (17). Substituting (56) into the integral 
on the left-hand side of (17), using the fact that ^[Qs,k,t{Z_,x}\ = Qs,fc,t(ifc_i,x) and 
substituting (56) into (15) gives 

% sup \Vs.k,t{ys\Vs-i-,Z_,x)Ays]<K^- 77^=I^^-c 77^- (57) 



inftezao(0 \i\Uei.ao{t) 



We now find a bound for Qs.k.t- By the definition of Qs,k,t in (13) and using the matrix 
norm inequality [Ax\i < K\\A\\sp(,c\\x\\ (|| • \\spec is the spectral norm), we have 



i=s+l 



k+s—i ( k+s—i ^ 

At+k+s-i ^ S W At+k+s-i-d \x 
r=l I d=0 ) . 



< 



K 



inftgzao(i) 



^ai{t + k + s) 



H+k+s-i ■ 



'fe-1 

At+k+s-i- 



spec 



To bound the above, we note that by Assumption 3.1(i), sup^g^ X]j=i — (1 ~ 
therefore there exists a 5, where Q < 5 < 5 < \ and such that, for all t, we have \\At+k+s-i x 
{Ilfco At+k+s-i-d}\\spec < K{\ ~ 5)^^^ for some finite K. Combining all of this gives 



Ss,fe,t(l fc_i,x) < . ^ ^ a,(t + fc + s) 



infigzao(0 



A 



t+k+s — i ' 



■k+s-i 

At+k+s-i-d 



spec 



(58) 



< 



K 

inftgzao(0 



Substituting the above into (57) gives (17). 

We now prove (18). We use the same proof to show (56), but apply (54) instead of 
(55) to obtain 

sup / supX>s,fc,t(2/s|ys-i,z,£)dys < r-f ttt sup Qs,fc,t(z, x). 
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By substituting the above into (16) and using the same proof to prove (17), we obtain 
/ 17/^(^0 sup <^ / s.wp'Ds,k,t{ys\ys-i-,z,x)dys \ dz 

(59) 

~ inftgzao(<) 

Since Qs,k,tiZ_,x) is a positive function and sup^.^^ Qs.k.t{Z_,x) = Qs,k.t{Z_,r]), we have 
]E[sup^g£ Qs,kAZ.,x)] < sup^g£E[Qs,fc,t(^,2)] = sup^g£ Qs,k,t{lk-i,^). Hence, by using 
(58); we have 

E[sup,^sQsMZ,x)] ^ Kil-6)''\\x\\ 



inftgzao(<) infigzao(0 
Substituting the above into (59) gives (18). □ 
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